zhurnaly.com

World Models vs Large Language Models

^z28th January 2026 at 9:54am

Gary Marcus's essay "Generative AI’s crippling and widespread failure to induce robust models of the world" (28 June 2025) is thoughtful, longish, semi-technical, and worth reading. In brief:

  • cognitive “world models” are “persistent, stable, updatable (and ideally up-to-date) internal representations of some set of entities within some slice of the world” — like our mental models of where we left our shoes, what we shouldn’t ever say to our spouses, how big the Earth is, etc
  • LLMs (mostly) don’t have “world models” — they just “extract correlations between bits of language”
  • that’s why LLMs “hallucinate”, why they can’t play chess (or tic-tac-toe) reliably, and why they can’t generally learn

(cf Genius and Complexity (1999-05-25), Intelligence Augmentation (2001-08-25), Mind Children (2003-04-17), Taxonomy of Machine Learning (2017-02-02), ...) - ^z - 2026-01-28

zhurnaly.com